The present invention relates to automated analysis and testing of hardware and/or software.
A software application or hardware device (collectively called a target device or a device-under-analysis) can be analyzed or tested in an automated way by using a second device called an analyzer. In this situation, the analyzer generates a test message (e.g., an invalid test message), delivers the test message to the target device, monitors the target device, and/or analyzes the monitored information to determine whether the target device is operating correctly. Analyses and test messages identify the limitations of a target device.
The analyzer can determine whether a target device was designed and implemented correctly by delivering various test messages to the target device and observing and analyzing its responses to the tests. For example, security analysis can be performed as described in U.S. patent application Ser. No. 11/351,403, filed on Feb. 10, 2006, the disclosure of which is hereby incorporated by reference in its entirety. The various tests may include valid test messages (called instrumentation vectors) to determine whether the device is still responding appropriately. For example, the analyzer can identify and characterize failures in a target device based on its responses to instrumentation vectors as described in U.S. patent application Ser. No. 11/760,600, filed on Jun. 8, 2007, the disclosure of which is hereby incorporated by reference in its entirety.
Observing a target device's responses to tests and instrumentation vectors may be insufficient in assessing (or analyzing) the target device's design and/or implementation. The tests may cause the target device to send invalid (or improper) messages to other devices, even while the target device is still responding properly to the invalid test messages and instrumentation vectors. This is especially problematic when the target device is a part of a larger system of multiple devices because while the target device itself may not fail, it may nevertheless negatively affect the health of the system in the course of processing the tests.
For example, in a network using Open Shortest Path First (OSPF) routing protocol, a router under attack may improperly communicate with other routers and corrupt their routing tables, link state databases (LSDBs), and/or other shared resources. The router itself may still respond to attacks correctly. However, the other routers with contaminated routing tables, LSDBs, and/or other shared resources may fail or malfunction as a result of the attacks. As another example, an enterprise system may include a web server hosting web applications connected with a database server. Attacks sent to the web applications may corrupt data stored in the database server, even though the web applications may still appear normal.
From the above, there is a need for a system and method to test and analyze a target device to ensure that invalid traffic would not cause the target device to negatively affect the health of a system of which the target device is a member.